Data Fields¶

const char * name
enum rte_member_setsum_type type
uint8_t is_cache
uint32_t num_keys
uint32_t key_len
uint32_t num_set
float false_positive_rate
uint32_t prim_hash_seed
uint32_t sec_hash_seed
float error_rate
float sample_rate
uint32_t top_k
uint32_t extra_flag
int socket_id

const char* name¶

Name of the hash.

Definition at line 205 of file rte_member.h.

enum rte_member_setsum_type type¶

User to specify the type of the setsummary from one of rte_member_setsum_type.

HT based setsummary is implemented like a hash table. User should use this type when there are many sets.

vBF setsummary is a vector of bloom filters. It is used when number of sets is not big (less than 32 for current implementation).

Definition at line 205 of file rte_member.h.

uint8_t is_cache¶

is_cache is only used for HT based setsummary.

If it is HT based setsummary, user to specify the subtype or mode of the setsummary. It could be cache, or non-cache mode. Set is_cache to be 1 if to use as cache mode.

For cache mode, keys can be evicted out of the HT setsummary. Keys with the same signature and map to the same bucket will overwrite each other in the setsummary table. This mode is useful for the case that the set-summary only needs to keep record of the recently inserted keys. Both false-negative and false-positive could happen.

For non-cache mode, keys cannot be evicted out of the cache. So for this mode the setsummary will become full eventually. Keys with the same signature but map to the same bucket will still occupy multiple entries. This mode does not give false-negative result.

Definition at line 238 of file rte_member.h.

uint32_t num_keys¶

For HT setsummary, num_keys equals to the number of entries of the table. When the number of keys inserted in the HT setsummary approaches this number, eviction could happen. For cache mode, keys could be evicted out of the table. For non-cache mode, keys will be evicted to other buckets like cuckoo hash. The table will also likely to become full before the number of inserted keys equal to the total number of entries.

For vBF, num_keys equal to the expected number of keys that will be inserted into the vBF. The implementation assumes the keys are evenly distributed to each BF in vBF. This is used to calculate the number of bits we need for each BF. User does not specify the size of each BF directly because the optimal size depends on the num_keys and false positive rate.

Definition at line 256 of file rte_member.h.

uint32_t key_len¶

The length of key is used for hash calculation. Since key is not stored in set-summary, large key does not require more memory space.

Definition at line 262 of file rte_member.h.

uint32_t num_set¶

num_set is only used for vBF, but not used for HT setsummary.

num_set is equal to the number of BFs in vBF. For current implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set summary. If other number of sets are needed, for example 5, the user should allocate the minimum available value that larger than 5, which is 8.

Definition at line 273 of file rte_member.h.

float false_positive_rate¶

false_positive_rate is only used for vBF, but not used for HT setsummary.

For vBF, false_positive_rate is the user-defined false positive rate given expected number of inserted keys (num_keys). It is used to calculate the total number of bits for each BF, and the number of hash values used during lookup and insertion. For details please refer to vBF implementation and membership library documentation.

For HT, This parameter is not directly set by users. HT setsummary's false positive rate is in the order of: false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit signature. This is because two keys needs to map to same bucket and same signature to have a collision (false positive). bucket_count is equal to number of entries (num_keys) divided by entry count per bucket (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_positive_rate is not directly set by users for HT mode.

Definition at line 294 of file rte_member.h.

uint32_t prim_hash_seed¶

We use two seeds to calculate two independent hashes for each key.

For HT type, one hash is used as signature, and the other is used for bucket location. For vBF type, these two hashes and their combinations are used as hash locations to index the bit array. For Sketch type, these seeds are not used.

Definition at line 305 of file rte_member.h.

uint32_t sec_hash_seed¶

The secondary seed should be a different value from the primary seed.

Definition at line 310 of file rte_member.h.

float error_rate¶

For count(min) sketch data structure, error rate defines the accuracy required by the user. Higher accuracy leads to more memory usage, but the flow size is estimated more accurately.

Definition at line 317 of file rte_member.h.

float sample_rate¶

Sampling rate means the internal sample rate of the rows of the count min sketches. Lower sampling rate can reduce CPU overhead, but the data structure will require more time to converge statistically.

Definition at line 324 of file rte_member.h.

uint32_t top_k¶

How many top heavy hitter to be reported. The library will internally keep the keys of heavy hitters for final report.

Definition at line 330 of file rte_member.h.

uint32_t extra_flag¶

Extra flags that may passed in by user

Definition at line 335 of file rte_member.h.

int socket_id¶

NUMA Socket ID for memory.

Definition at line 337 of file rte_member.h.